Integrating Adapters with Calibo Accelerate Orchestrator Agent

Security policies of customer organization may not allow the core services of a third-party platform like Calibo Accelerate to have direct access to tools and technologies deployed in customer's cloud account. The Calibo Accelerate platform needs to successfully connect to customer's existing user accounts for cloud services, containers, development tools, software frameworks, code and artifact repositories and other technology stack deployed in their cloud environment. Calibo Accelerate needs to consume credentials, API keys, tokens, and other secrets for users to connect to tools from within the platform. As you develop adapters to add more tools and technologies into the Calibo Accelerate platform, you must also understand the architecture, configuration and uses of Calibo Accelerate Orchestrator Agent in this integration framework. The Calibo Accelerate Orchestrator Agent helps you establish a secure connection between the tools deployed in a private cloud network and the Calibo Accelerate platform.

Why Calibo Accelerate Orchestrator Agent?

Calibo Accelerate Orchestrator Agent is a crucial component that acts as the intermediary between the Calibo Accelerate platform and the customer's own set of tools. Its primary function is to facilitate the secure communication between Calibo Accelerate services and customer's tools residing in their private network.

Components of Calibo Accelerate Orchestrator Agent

The following table contains the component microservices of the Calibo Accelerate Orchestrator Agent and their functions. All these microservices together are responsible for resolving tools secrets from your secrets management tool.

Component	What it Does
REST Agent Service	This is a thin client which forwards the REST API requests received from the Calibo Accelerate platform to tools deployed in your cloud environment.
Management Service	This microservice is responsible for registering the Calibo Accelerate Orchestrator Agent with the Calibo Accelerate platform. It validates the API key before processing each REST API request coming from the Calibo Accelerate platform. It is also responsible for the regeneration of the API key. See API Key Validation for Calibo Accelerate Orchestrator Agent.
Data Service	This microservice helps the Calibo Accelerate platform connect to data tools such as Databricks, Amazon S3, AWS Glue, RDBMS, and Snowflake, among others. See Tools Supported by Calibo Accelerate Orchestrator Agent.
DevOps Service	This microservice helps the Calibo Accelerate platform connect to DevOps tools such as Jenkins and Kubernetes.

Calibo Accelerate Orchestrator Agent Architecture

How Calibo Accelerate Orchestrator Agent Works

Customers can host their tools within a Calibo-managed cloud account, or within their own private network. When configuring a tool, various details such as its URL and associated secrets (including IDs, passwords, or tokens) are provided. If a tool is openly accessible on the network or hosted within the Calibo Accelerate environment, establishing connections is easy.

However, complexities may arise when a tool is hosted within the customer's private network and is not publicly accessible. In such scenarios, customers can use the Calibo Accelerate Orchestrator Agent, which comprises a set of essential microservices. These microservices are deployed directly within the customer's cloud account, facilitating seamless communication between the Calibo Accelerate platform and customer's tools.

To ensure smooth orchestration, the Calibo Accelerate platform continually monitors the Orchestrator's status by receiving regular heartbeats, ensuring that all services remain operational. When configuring a tool hosted in this manner, users can opt to access it via the Calibo Accelerate Orchestrator Agent. To do this, users must select the Orchestrator from the available list and proceed with the tool's configuration.

Customers must securely store their tool credentials within their own secrets management tool, which can be one of several trusted options like AWS Secret Manager, Azure Key Vault, or HashiCorp Vault, among others. During the tool's configuration process, customers are required to provide the keys necessary to access these stored secrets, thereby enabling secure and authorized access to their tools, even within a private and restricted network environment.

Required Changes to Integrate Adapters with Calibo Accelerate Orchestrator Agent

An adapter that you build to integrate new tools and technologies with the Calibo Accelerate platform is mainly responsible for establishing the connection between the tool or technology and the Calibo Accelerate Orchestrator Agent. You may need to make some code changes in the components of Calibo Accelerate Orchestrator Agent for this integration.

Add the respective dependency to gradle file of the respective agent service Data Agent or the DevOps agent. The dependency is added as a JAR file in the agent artifact.

Build the agent service, package, and create a new release of Calibo Accelerate Orchestrator Agent.

The Calibo Accelerate platform automatically detects the new available Orchestrator deployment and generates a new set of instructions on the UI to deploy the services in the customer's private cloud environment.

The following table contains the expected changes in the Calibo Accelerate Orchestrator Agent code for integration with an adapter.

Component	Name of Service	Impact
REST Agent Service	plf-orchestrator-service	No impact
Management Service	plf-management-orchestrator-service	No impact
Data Service	plf-common-orchestrator-service	Code changes are required in the Calibo Accelerate Orchestrator Agent. For example, see Integrate a New Technology in an Existing Crawler Category.
DevOps Service	plf-devops-orchestrator-service	Code changes depend on tools being integrated through an adapter. If any custom logic is required, you may need to make minor changes within the DevOps Agent code.

Know more about the Calibo Accelerate Orchestrator Agent with the help of the following topics: